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Abstract 

We discuss the motivation and an architectural framework 
for using small mobile robots as automated aids to 
operators of nondestructive inspection (NDI) equipment. 
We review the need for aircraft skin inspection, and 
identify the constraints in commercial airlines operations 
that make small mobile robots the most attractive 
alternative for automated aids for NDI procedures. We 
describe the design and performance of the robot (ANDI) 
that we designed, built, and are testing for deployment of 
eddy current probes in prescribed commercial aircraft 
inspections. We discuss recent work aimed at also 
providing robotic aids for visual inspection. 


L Background 

Our goal is to replicate and enhance the capability of 
aircraft skin inspectors who use hand-held instruments 
(and their own senses and intelligence) to detect and 
classify flaws in aging aircraft. Our underlying concept is 
to use mobile robots, automated control, and automated 
interpretation of sensors and instruments to make difficult 
measurements in difficult environments. Potential 
application area include not only airplane skins, the 
subject of this paper, but also problems such as bombs in 
luggage, contraband in cargo containers, verification of 
disarmament treaty compliance, characterizing 
environmentally contaminated sites, and a variety of 
manufacturing problems, e.g., measuring composition 
gradients in large process tanks, transportation problems, 
e.g., bridge inspection, and scientific research problems, 
e.g.! checking the integrity, alignment, etc, of large 
instruments such as radio telescopes and particle 
accelerators. These few examples just begin to suggest 
the universe of potential application areas and specific 
applications. A general hierarchical paradigm for 
organizing the common issues of measurement, 
manipulation, mobility, and monitoring characteristic of 
all these problems is illustrated explicitly for the aging 
aircraft problem in Figure 1. 



Figure 1: The 4M-s of automation 
for aircraft inspection. 

II. Inspection of Aging Aircraft 

Aircraft skins inflate and deflate with each cycle of 
pressurization and depressurization. The resulting stress 
causes several kinds of damage, primarily radial cracks 
around rivets, delamination of skin joints, and subsurface 
cracks in the structural members to which the skin is 
attached. Delamination is exacerbated by corrosion, 
which is particularly prevalent in warm moist climates. 
Cracks and corrosion, accelerated by island-hopping 
operation, resulted in April 1988 in a large section of skin 
tearing off the lop of the fuselage of an Aloha Airlines 
Boeing 737. The resulting press coverage of the 
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airplane’s seemingly miraculous safe landing brought 
these problems prominently to the attention of the public, 
and resulted in an aggressive prevention, detection, and 
remediation program by aircraft operators in close 
cooperation with each other, the aircraft manufacturers, 
and the FAA 1 - 2 ’ 3 . Structural effects of aging in other 
areas, such as engines, fuel tanks, landing gear, etc, 
possibly will be the subjects of future automation 
research, but for the present our program is concentrating 
on skin and the immediate supporting substructures. 

Through programs of periodic inspection of known 
problem areas on each aircraft type, skin cracks and 
corrosion are typically found well before they reach 
hazardous size. The problem areas are specified by 
service bulletins" issued by aircraft manufacturers, and 
by "airworthiness directives" issued by the FAA. 
Compliance with airworthiness directives is mandatory. 
Compliance with service bulletins is at the airline 
operators discretion, but we are told that in practice they 
are treated as mandatory. 

About 90% of skin inspection is visual, by inspectors 
trained for the task, most of the remainder is by eddy 
current probes, and a fraction of a percent is by other 
instrumentation of which the best known is probably 
ultrasonic. Our program is focused in its initial phases on 
automation as an aid to skin inspection using eddy current 
probes. Initially we will use machine vision to aid probe 
placement and robot navigation and to update the 
navigation database with descriptions of patches and other 
deviations from "as designed". We are beginning to 
investigate automated aids to visual inspection via a new 
program in which a small limited functionality robot will 
be used deploy 3D-stereoscopic cameras. Working with 
experienced visual inspectors, we will evaluate the 
acceptability of computer aided remote (teleoperated) 
visual inspection. 


Eddy Current Inspection 

The eddy current method 4 uses a transmitting coil and a 
receiving coil (they may physically be one coil) coupled 
electromagnetically through the metal under inspection. 
Eddy current probes vary in tip area from several square 
centimeters to about one square millimeter, obviously 
trading off decreasing areal coverage for increasing 
sensitivity to small flaws as the size decreases. 
Anomalies in the impedance that characterizes the 
coupling indicate cracks, corrosion thinning, and other 
flaws. Inspectors generally watch an x-y oscilloscope 
display whose x-axis represents the in-phase (resistive) 
part of the impedance and whose y-axis represents the 
quadrature-phase (inductive or capacitive) part of the 
impedance. Figure 2 illustrates a probe, and Figure 3 


illustrates typical impedance plane signals. The 
inspectors compare patterns traced out on the screen when 
the probe is passed over a potential flaw with the pattern 
traced out when the same probe is passed over a 
calibration standard manufactured with a machined flaw 
in the simulated local structure. The probe geometry, 
operating frequency, scan path, etc, are chosen to 
optimize sensitivity to each anticipated flaw. High 
operating frequencies are attenuated in a short distance, 
and thus probe only the surface. Low operating 
frequencies penetrate deeper, and in some geometries can 
penetrate the skin entirely and probe for cracks in the 
supporting framework. Under typical operating 
conditions power levels are sufficiently low that the 
method is extremely linear, so it is possible to operate a 
probe with a composite multi-frequency transmitted 
waveform and to separate electronically the high- 
frequency surface-sensitive received signal components 
from the low-frequency substructure-sensitive 
components. 



Figure 2: "Reflectance” or "pitch-catch” 
eddy current probe. 

Modem eddy current systems can be set to alarm on 
traces that enter or fail to enter preset rectangular 
windows in complex impedance space. Initially we will 
rely on these alarms to alert the inspector to potential 
flaws indicated by anomalous signals. These areas will be 
marked for easy identification by the inspector, e.g., by 
daubing suspect rivet heads with a washable paint. 
Pattern recognition integrated with rule based systems is 
an accepted method for automating interpretation and 
classification of eddy current signals in other applications, 
e.g., inspection of heat exchanger tubes in nuclear power 
plants 5 . Neural network methods have been similarly 
successful in similar applications^. As the program 
progresses we will add additional software to implement 
promising approaches to automated and improved eddy 
current signal interpretation and classification. 
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Figure 3: Eddy current signals 

in the complex impedance plane. 


m. Automated NonDestructive Inspector 

We considered many approaches to automation-assisted 
eddy current probe deployment, with three primary 
variants: the gantry-based "car wash”, the vehicle-based 
"cherry picker", and the self-contained "window washer". 
The pros and cons of these alternatives have been 
discussed in detail elsewhere 7 - 8.9,10,11,12,13,14,15. in 
summary, the "window washer" design that we eventually 
chose for the system is dictated by the pragmatics of 
fitting NDI nondisruptively into the flow of passenger 
aircraft maintenance operations. These constraints 
suggest a small (under one meter maximum dimension) 
mobile platform that is able to walk or crawl over most if 
not all of the aircraft skin, whatever its orientation. This 
capability we achieve with active (vacuum assisted) 
suction cups. A concept sketch for the resulting robot 
(ANDI, the Automated NonDestructive Inspector) is 
shown in Figure 4. It is not the easiest approach, but it is 
the most acceptable, and incidentally it is the approach 
that requires the most interesting enabling research. 


Cruciform Design 

Because there is generally more fore-aft than 
circumferential inspection path, the robot is designed with 
a cruciform geometry that enables it to move along fore- 
aft paths most rapidly; this results in a design in which it 
moves on circumferential paths somewhat more slowly, 
and in skew directions adequately, but a bit awkwardly. 
The mechanical design is sketched in Figure 5 and shown 
close-up (with eddy current probe in the foreground and a 
graphical depiction of the probe output on the computer 


Aircraft Skin Inspection Robot 



Figure 4; Concept sketch for ANDI. 

monitor screen in the background) in Figure 6. It has 
many features in common with the class of mobile robots 
known in the literature as "beam walkers" 16 ’ ,7 . However 
unlike most beam walkers our robot is able to side step 
almost as easily as it can walk forward or backward. The 
two cross members ("bridges") are normally locked at 
right angles to the main longitudinal member ("spine"), 
but they can be released to pivot freely by about 15° in 
either direction; this permits the robot to steer and thus to 
travel along paths that are neither strictly fore-aft nor 
strictly circumferential. Pneumatically actuated up-down 
degrees of freedom on the four suction cups at the ends of 
the bridges enable the walking motion, and another 
pneumatic actuator enables the raising and lowering of the 
eddy current probe. The sliding motions of the bridges 
along and perpendicular to the spine are actuated by 
electric motors. 
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Figure 5: Mechanical features of ANDI, 

showing four camera mounting points. 
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Figure 6: ANDI, showing eddy current probe 
and its signal on the computer monitor. 


Alternative Designs 

It is regrettably easy to confuse ANDI, particularly given 
its multiply anthropomorphic name**, with the much 
larger and more complex system of which it is essentially 
just the mechanical end effector. It is thus appropriate to 
emphasize explicitly that ANDI is just the first prototype 
mechanical end effector of a large and complex system 
(most of which is black boxes full of electronics and 
computers) that can accommodate many different end 
effectors. ANDI is designed to demonstrate the feasibility 
of using robots to assist inspectors of aging aircraft. But 
ANDI is not the last end effector that will ever be needed 
for this task. There are places on an airplane skin where 
ANDI cannot adhere, e.g,, sharply curved regions around 
the nose, tail, and leading and trailing edges of the wings 
and horizontal and vertical stabilizers. There are places 
where ANDI can adhere but may turn out to be 
insufficiently agile to deploy the eddy current sensor in an 
effective pattern, e.g., perhaps around doors, windows, 
repair patches, etc. Our goal is to demonstrate what 
ANDI can do. We guess it can do something like 80% of 
the mandated and recommended eddy current inspections 
on DC -9 or Boeing 737 and larger aircraft. If ANDI 
proves its technical and economic worth in these 
applications, we are confident that we (and others!) will 
be able to design as many specialized mechanical end 
effectors as are needed to cover the applications ANDI 
cannot. 

A block diagram of the currently envisioned complete 
system is shown in Figure 7. 



Figure 7: Block diagram of planned complete system. 
Special Purpose Actuators 

Our initial eddy current sensor deployment demonstration 
is targeted on part of a mandated inspection on the 
fuselage of a DC-9 that uses a "reflectance** or "pitch- 
catch” probe with mechanically independent but 
electrically coupled transmit and receive coils. This 
particular inspection has both a surface crack and a 
subsurface crack component which we address 
simultaneously by composite dual frequency operation. 
The reflectance probe geometry Is sensitive to integrated 
conditions over a fairly large patch of skin (a few square 
millimeters), so it is forgiving of small errors in 
placement relative to the rivets under examination. Under 
these circumstances it is adequate to deploy the probe 
with a simple up-down lifter mechanism and let it self- 
align with the skin under the influence of a constant-force 
spring. Another part of the planned demonstration 
inspection uses a "pencil” probe with a single coil that has 
a much smaller sensitive area. It must thus be placed and 
scanned more accurately, e.g., along a path that is 
tangential to each rivet, which may require closed loop 
guidance. Another small but necessary part of the 
demonstration inspection requires moving a pencil probe 
completely around the circumference of several rivets. 
The more complex probe paths that this inspection 
component requires can In principle be achieved by 
coordinating the motions of bridge-along-spine and 
bridge-perpendicular-to-spine, but we anticipate that 
obtaining the necessary mechanical precision and 
simplicity of control may require adding some nominally 
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redundant special purpose degrees of freedom, e.g., a 
rotary mechanism for precisely circumnavigating 
individual rivets. 

Path Control 

The path control system addresses mechanical positioning 
of the eddy current probe and the robot at four distance 
scales corresponding to the tasks of alignment, guidance, 
navigation, and path planning. 

Alignment means the relative position of the eddy current 
or alternative probe and the rivet or other component 
under inspection. The inspection protocol is predicated 
on the assumption that the probe will be moved along a 
precise short path relative to the part geometry. Signal 
classification can be done meaningfully only if this path is 
followed. 

Guidance means, for rivet inspection, moving the probe 
from one rivet to the next and arriving there in correct 
alignment. For other inspections, e.g., for corrosion 
somewhere along a skin joint, it means following the 
required inspection path. In this case it is differs from 
alignment only in distance scale. 

Navigation means coordinating walking and probe 
guidance so that an inspection that spans multiple robot 
steps proceeds smoothly and certainly. 

Path planning means being able to traverse as rapidly as 
possible, without inspecting, long distances between areas 
that require inspection. This is the scale at which 
collisions with undocumented parts of the airplane (e.g., a 
non-standard antenna), expected parts in an unexpected 
state (e.g., an access hatch left open during maintenance), 
and other maintenance equipment (e.g., a wrench left on a 
bolt head) are potentially serious problems. 

We expect to achieve the necessary position accuracy by 
dead-reckoning using high mechanical precision motion 
over short distances between map database landmarks and 
using machine-vision-based correction at each landmark. 
The obvious landmarks are the rivets themselves, each of 
which is in principle individually identifiable in the 
aircraft design database. The eddy current signals 
themselves then provide an additional and perhaps finer 
level of correction: misalignment signatures are 

recognizable and quantifiable, although some sign 
ambiguities would have to be resolved by active sensing. 
Skin joints and skin joint intersections provide additional 
landmarks. They are particularly appropriate for 
navigation and path planning, in contrast with the rivets, 
which are particularly appropriate for alignment and 
guidance. Skin joints are farther apart than rivets, a 
disadvantage in terms of dead-reckoning error 
accumulation, but their existence is more consistent from 


airplane to airplane (of the same type) since their 
locations are less likely to be changed by modifications 
and repairs. The skin joints and skin joint intersections, 
referenced in terms of the underlying longeron (or 
stringer) and spar (or body station) identification 
numbers, are in fact the features in terms of which 
mandated and recommended inspections are defined. 

In principle the map databases are all on-line at the 
factory for as-designed and as-built, and on-line at the 
hangar for as-modified and as-repaired. In practice the 
data are still on paper for all aircraft except the generation 
now in gestation, e.g., the Boeing 777, and we expect we 
will have to use ANDI to bootstrap populating its own 
map and exception database. 


Vision System 

ANDI will have at least four cameras in the alignment, 
guidance, navigation, and path planning system. Cameras 
will also have roles in visual flaw detection, but not until 
a later phase of the program. 

Macro Camera 

The first camera will be mounted on the same platform as 
the eddy current probe, with a macro capability giving it a 
field-of-view of approximately one rivet. It will be used 
for fine alignment and for the inspector’s visual 
observation of the appearance of the rivet and the adjacent 
skin at high magnification. In some inspections that 
require precision probe alignment the alignment control 
loop may incorporate the eddy current sensor signal as 
well. In later phases image understanding will be 
incorporated for visual flaw detection, and possibly as an 
adjunct to eddy current or other NDI probes. For 
example, a particular eddy current probe that has a 
radially symmetric field geometry may sensitively 
indicate the presence of a radial crack, but will obviously 
be blind to its orientation; it may then be useful to use the 
high magnification camera to find its orientation. 

Alignment Cameras 

The second and third cameras, each with medium 
magnification fields of view of about 10 cm x 15 cm, or a 
line of four to six rivets, will be mounted at the head and 
tail of the spine. These cameras will be used to locate line 
segments of rivets. A robust best-fit'* to the head and tail 
line segments will guide the eddy current sensor along a 
scan line. This guidance functionality is required early, 
so these will be the first cameras installed on ANDI. 
Guidance is actually required before alignment, because 
the initial eddy current probe is of the "pitch-catch" or 
"reflectance" type, which is sufficiently tolerant of 
misalignment that little alignment (fine adjustment about 



the guidance line) is likely to be needed. The imagery 
from these cameras will also be made available to the 
inspector for opportunistic flaw detection of, for example, 
lightning holes and small dents. As in the case of the high 
magnification camera, automation of the flaw detection 
role for these cameras will come in a later project phase. 
However, we have already made substantial progress 
prototyping the computer vision based automation of the 
alignment function. Several rivet finding algorithms have 
been tried, including edge detection followed by region 
growing, gray level variance, and a trained neural 
network, yielding the general conclusion that even with 
uncontrolled lighting, low contrast, and interference from 
specular reflectances, any scale-sensitive operator that has 
the actual rivet size hard wired into it will succeed. A 
conventional robust line-fitting algorithm based on 
minimizing the mean absolute deviation almost always 
correctly draws the desired line through three, four, or 
five rivets even for the most ghastly poor images. Early 
results are discussed and illustrated in the following 
section. 

Zoom Lens Camera 

The fourth camera, with an ordinary zoom lens’s range of 
focal lengths and working distances, will be mounted on a 
motorized pan-tilt head high above ANDI’s tail end. In 
the initial experiments, before general purpose navigation 
and path planning algorithms are in place, ANDI will be 
teleoperated between inspection stations. Thus the fourth 
camera will initially be the inspector’s eye on the robot’s 
actual configuration, possible interferences or collisions, 
sensible paths between inspection stations, and gross 
visual flaws, e.g., pillowing due to extensive subsurface 
corrosion. As the program progresses machine vision 
algorithms will increasingly use this camera for 
proprioception (visually confirming that the actual robot 
pose corresponds to the control system’s model of the 
pose), collision avoidance and footfall decisions (new 
radio antennas, skin patches, or raised head replacement 
rivets will have to be found, avoided, and entered into the 
database), long distance path planning between inspection 
stations, and opportunistic detection of large flaws. 

Vision Based Alignment 

While there are important exceptions that we will 
eventually have to address, most of the time rivets line up 
neatly in evenly spaced rows and columns. ANDI is 
designed to take maximum advantage of this design rule: 
what ANDI can do most effortlessly and precisely is to 
scan an eddy current sensor along a straight line segment 
parallel to and almost the full length of its spine. The 
essential alignment problem is thus to align the spine 
parallel to the line segment under inspection. The 


approach we are developing is to best-fit visually a short 
line segment near each end of the spine, best-fit the long 
line segment to the two short line segments, and scan 
along the long line segment open loop unless the eddy 
current data show features that suggest the rivet line 
wiggles enough that transverse corrections are needed. 

On the assumption that if a computer vision algorithm 
works well with terrible looking images it will probably 
work better with better looking images, we developed our 
approach on a sequence of images that we collected with 
uncontrolled lighting, uncontrolled surroundings (which 
are obvious in specular reflection), poorly controlled 
camera standoff from the riveted surface (a test panel with 
a radius of curvature and other features comparable to a 
Boeing 737 or DC-9), and a consumer grade 8 mm 
camcorder camera that we scanned over the test panel by 
hand. We digitized to 8 bits x 3 colors about 80 frames 
grabbed from the tape at about 1.5 sec intervals. Each 
frame was digitized into 480 pixels x 512 lines x 3 colors, 
then averaged in 8 x 8 blocks into 60 pixel x 64 line x 3 
color working images. At the lowered resolution the rivet 
line segment finding pipeline runs at approximately real- 
time (1.5 sec/frame) on a workstation. With the camera 
parameters and resolution we used, rivets are circular 
blobs that generally fit into a 7 x 7 block. A typical 
frame, with gray levels computed by averaging the RGB 
values, is shown in Figure 8. 



Figure 8: Raw image showing a line of rivets. 
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Conventional Algorithm 

As mentioned in the previous section, finding rivets is 
easy even when the images are as ugly as this one: any 
sensible operator with a scale length matched to the rivet 
size works fine. Under these circumstances a useful 
strategy is to choose an operator that rarely misses a real 
rivet even at the price of occasionally finding a false rivet, 
provided that one or more downstream modules can be 
tailored to reliably reject false rivets. Finding all real 
rivets plus some non-rivets we can in fact do with a 
Canny edge detector 19 . Next we observe that specular 
reflections, the main potential source of false rivets, look 
different in each of the three color bands, whereas real 
breaks in the metal, e.g., rivet edges, have a generally 
neutral hue. Thus in the second image processing step we 
reject most of the non-rivets by fusing the three color 
bands, retaining only those pixels tagged by the edge 
detector separately in each band. The result is shown in 
Figure 9. 



Figure 9: Rivet edge detection by the Canny operator. 


Next a region-growing ("grass-fire” 20 ) algorithm 
transforms the perimeters found by the edge detector into 
blobs filling the areas of the rivet heads. Blobs are 
rejected if they fail to meet simple geometrical criteria for 
rivets, e.g., area, aspect ratio, and fill factor within 
heuristic numerical bounds. The centroids of the 
surviving blobs are then used as input to a robust 
(insensitive to outlier) line fitting algorithm 18 . The result 
is shown in Figure 10. 



Figure 10: Line fit to the five rivets found. 


Neural Network Algorithm 

An alternative algorithm was built by training a neural 
network simulated in software. This method approaches 
the problem by saying that rather than discovering and 
finding suitable discrimination parameters and then- 
ranges intuitively, relevant parameters and ranges can be 
found mechanically by systematically modifying the 
parameters of a generalized input-output network until it 
reliably behaves as a rivet/not-rivet classifier when 
applied to an operator-classified training set 
representative of the problem; if the training set is 
adequately representative of the problem domain then the 
trained network will also be able to classify rivets and 
not-rivets that were not in the training set. 

To implement this method we constructed and trained 
(using the back-propagation algorithm 21 ) a three layer 
neural net with an input layer consisting of 147 units (a 7 
x 7 retina in each of three color bands), five hidden units, 
and one output unit whose binarized output we interpret 
as "rivet" and "not-rivet". The network was trained on 40 
frames and tested on 40 different frames. Figure 11 
shows the output of the trained neural network operator: 
bright areas are "rivet-like", dark areas are "not-rivet- 
like". Figure 12 shows the result of thresholding and 
extracting connected regions of this image, and also the 
performance of the robust line fitting algorithm on the 
result. 
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Figure 11: Rivet image found by the neural network. 



Figure 12: Line fit to the 
rivets found in the neural network image. 


Inspector’s Workstation 

Figure 4 depicts an inspector’s workstation adapted to the 
environment and culture of aircraft maintenance and 
inspection. During ANDI’s laboratory research and 
development phases the interim implementation of this 
workstation is based on two 80486-based PCs. One 
supports the inspector’s mouse-and-menu-based interface, 
serial communications with the motor controllers, and a 
general purpose data acquisition and control system with 
multiple analog-to-digital inputs, digital-to-analog 


outputs, and digital input and output lines for interacting 
with various sensors (e.g., suction cup vacuum) and 
actuators (e.g., solenoid valves controlling pneumatic 
cylinders). The second PC supports the eddy current 
probe system and its display. The interim vision system 
is on an independent proprietary computing platform. It 
now supports alignment of the robot spine with rivet lines. 
Its permanent successor will support the additional vision 
system requirements outlined above. 

As development continues the multiple platforms and 
displays will be rationalized. Our aim is to distribute 
processing power (which will include providing ANDI 
with on-board computing power for pose and gait control, 
etc)," and to coalesce the multiple displays by using a 
powerful windowing system to give the inspector access 
to controls, signals, images, and data on a single screen. 
A rudimentary interim database is in place for this 
function during laboratory tests of vision based alignment, 
eddy current sensor scanning, and navigation during 
walking between scanned locations 

Database and Archiving 

Aircraft skin inspections are now pass/fail. There is no 
requirement to record anomalies the reporting threshold. 
In practice, we are told, airline operators repair all 
detected flaws, even those below the mandatory and 
recommended thresholds. Even if this were not the case, 
pass/fail recording would not necessarily be risky: the 
thresholds have substantial safety margins, and there are 
good growth models 22 for predicting how far in the future 
will repair be necessary. These encouraging practices and 
circumstances notwithstanding, we nevertheless expect 
that the predictive capabilities that will follow database 
archiving and statistical analysis of quantitative inspection 
results will facilitate maintenance scheduling and 
potentially increase safety. Thus an on-line distributed 
database, with an architecture open to access from 
multiple potential inspection and maintenance locations 
and tools for trend analysis, improved statistical 
predictions, and pattern discovery is an integral part of 
our program. We envision a hierarchical architecture 
with aircraft type at the top, followed by production 
series, customer configuration, fleet-wide modifications, 
and, on an airplane-by-airplane basis, records of 
individual modifications and repairs. These include 
individual functional modifications, repair patches, plated 
regions, regions with oversize replacement rivets, etc. 
These structural features need to be documented for robot 
navigation as well as for maintenance. 
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IV. Visual Inspection 

As mentioned earlier, close to 90% of aircraft inspection 
is visual; our choice of eddy current inspection for the 
first demonstration of automation to aid aircraft inspectors 
was driven by the relative simplicity of automating 
deployment of eddy current probes (and NDI probes in 
general) in comparison with visual inspection. Unlike 
NDI, where the goal is usually to detect a flaw whose 
location and nature is known in advance (from previous 
experience or from computer modelling), visual 
inspection has a substantial opportunistic component. 
The visual inspector’s goal is to find not only the 
anticipated failures, but "everything else" as well: dents, 
lightning strikes, and other kinds of damage of an 
unpredictable nature in unpredicatable locations. The 
open-ended quality of this task makes it an unlikely 
candidate for a level of automation approaching the level 
we are planning for NDI. 

However discussions with airline management and NDI 
inspection personnel suggest that an integral visual 
inspection capability may be perceived as an 
indispensable component of any economically viable 
system of automated aids to NDI. 

In response to this perception, a mobile end-effector like 
ANDI does suggest itself as a teleoperable platform from 
which ground-based visual inspection might efficiently be 
conducted. If this could be accomplished, it would be 
valuable for many of the same reasons that ground-based 
NDI is valuable: reduced set-up time, human-factors 
issues of inspector performance in a difficult 
environment, inspector safety, database access, data 
archiving, etc. The question is whether remote cameras 
can provide sufficiently high quality (presumably 
meaning primarily high resolution) imagery to satisfy the 
notoriously fussy (we are comforted to say) visual 
inspectors. We recently began a program whose goal is to 
answer this question. This program combines elements of 
the FAA-sponsored ANDI project with salient elements 
of an ARPA-sponsored project in 3D-Stereoscopy 
Technologies for image and graphics visualization. 

One of the costs of human inspection is attributable to the 
difficulty of safely getting the inspector to the right place 
on the airplane: it involves erecting scaffolding, providing 
safety harnesses, etc, all of which can take more time than 
the inspection per se. ANDI can be placed on an airplane 
fuselage at human chest level, and directed to move to 
any area requiring inspection without erecting scaffolding 
and without endangering the human inspector. Thus even 
a teleoperated capability, with only the most rudimentary 
elements of automation (e.g., computer coordination of 
gait), could permit the inspector rapidly and safely to 
perform the necessary visual inspections. With 


appropriately selected cameras and actuators thus could 
clearly be done at the required variety of points-of-view, 
magnifications, lighting conditions, etc. 

3D-Stereoscopic Vision 

We are particularly interested in the prospect of providing 
the visual inspector with binocular 3D-stereoscopic 
vision. Stereoscopic perception appears to be important 
to the visual inspectors who we have observed on the job. 
We speculate that this may be because of its importance 
both in perceiving and in rejecting the effects of specular 
reflection off the mirror-like aircraft skin. Specular 
reflection appears to be important to inspectors looking 
for the presence or absence of specific flaws: they often 
move their heads and lights as the look for an expected 
tell-tale glint. Specular reflection is particularly apparent 
in binocular 3D-stereoscopic imagery because the sharply 
directed reflection appears much brighter in one or the 
other image, in contrast to the diffuse reflections, whose 
intensities are evenly balanced in the two images. [For 
this reason waterfalls and fast running streams, which are 
notoriously difficult to photograph (and paint) well, look 
spectacularly realistic in 3D-stereoscopic imagery.] 
Furthermore, the depth perception provided by 3D- 
stereoscopic imagery also makes it easy to reject artifacts 
of the environment that are reflected by the aircraft skin. 
Without depth perception it is impossible to know (except 
by high-level knowledge of the context) whether features 
of the imagery are in the skin or in the environment and 
seen in reflection. With depth perception, image features 
that are not in the plane of the skin can be rejected 
straightforwardly, even automatically. 

The components required in a 3D-stereoscopic system are 
(1) a matched pair of cameras (analogous to the human’s 
two eyes), (2) suitable and suitably controllable lighting, 
and (3) a display that is capable of directing the image 
corresponding to the right camera to the operator’s right 
eye and the image corresponding to the left camera to the 
operator’s left eye. There is no ideal way to accomplish 
(3). Special video taping equipment is also needed if the 
imagery is to be recorded. A variety of commercially 
available solutions have pros and cons that we are 
evaluating in context of the visual inspection application. 
Available solutions include frame, field, and subficld 
sequential methods with active shuttering eyewear, field 
and sequential methods with interline-polarization and 
passive eyewear, and "virtual reality" approaches using 
head-mounted displays. We have one subfield sequential 
and one interline-polarization system operational, so will 
conduct our initial experiments with these systems. 

We are building for these experiments a simple mobile 
manipulator that will move over an airplane panel test 
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surface according to the inspector’s instructions mediated 
by a computer that will support a suitably high-level 
interface. A simple mobile manipulator (in contrast to 
ANDI) will suffice because (unlike ANDI) it will have to 
operate, for these evaluation experiments, only on a more- 
or-less horizontal surface. 
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